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Abstract 

Sensitivity analysis of the optimal solution in response surface methodology is stud- 
s' I ied and an explicit form of the effect of perturbation of the regression coefficients 
V^ ■ on the optimal solution is obtained. The characterisation of the critical point of 
the convex program corresponding to the optimum of a response surface model is 
also studied. The asymptotic normality of the optimal solution follows by standard 
methods. 

1 Introduction 

From a point of view of the mathematical programming, the sensitivity analysis studies the 
effect of small perturbations in the parameters on the optimal objective function value and 
on the critical point for mathematical programming problems. In general, these parameters 
shape the objective function and constraint the approach to the problem of mathematical 
programming. The sensiti vity analysis of the mathematical prog ram ming has been stud- 
i ed by several authors, see Ijagannathani (|l977l) . iDupacoval (|l98J) and iFiacco and Ghaemi 



(|1982| ) among many other. As a direct consequence of the sensitivity analysis, the asymptotic 
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normality study of the critical point follows by standard methods of mathematical statis 



tics (s ee similar results for the case of maximum likelihood estimates lAitchison and Silve' 



([19581)). This last consequence makes the sensitivity analysis very appealing for researchinj 



from a statistical point of view. 

In this work, we study the effect of perturbations of the regression parameters on the 
optimal solution of the response surface model and then the asymptotic normality of the 
critical point is obtained. 

A very useful statistical tool in the study of designs, phenomena and experiments, is the 
response surfaces methodology. Which enables to find an analytical relationship between 
the response and controlled variables through a process of continuous improvement and 
optimisation. 

Specifically, it is assumed that a researcher knows a system, which considers some ob- 
servable response variables y which depends on some input variables, xi, . . .x^. Also it is 
assumed that the input variables x^^ can be controlled by the researcher with a minimum 
error. 

In general we have that 

y{^)=-q{xi,...Xk) (1) 

where the form of the function 77(-), usually termed as the true response surface, is unknown 
and perhaps, very comphcated; and x = (a;i, . . . xu)' ■ The success of the response surfaces 
methodology depends on the approximation of ?7(-) for a polynomial of low degree in some 
region. 

In this work we assume that •q{-) can be soundly approximated by a polynomial of second 
order, that is 



V~^ n . V~^ ^ 9 . v~^ v~^ 



y(x) = /3o + 2^ P^X^ + 2^ I3uxi +2^}^ |3^jX^Xj (2) 

i—1 i—1 i—1 j>i 

where the unknown parameters /?■'* can be estimated via regression's techniques, as it is 
described in next section. 

Next, we are interested in obtaining the levels of the input variables a;/ such that the re- 
sponse variables y is minimum (optimal). This can be achieved if the following mathematical 
program is solved 

min j/(x) 

X 

subject to (3) 

X e X, 

where X is certain operating region for the input variables a;/. 
Now, two questions, intimately related, can be observed: 

1. When the estimation of ([2]) is considered into ([3]) the critical point x* obtained as 
solution shall be a function of the estimators /3„ ^ of the /3-^. Thus, given that /3 * are 
random variables, then x* = x*(/? ^) is a random vector too. The question is, if the 
distribution of /3 is known, then what is the distribution of x*(/3 '')? 

2. And, perhaps it is not sufficient to know only a point estimate of x*(/3- *), should be 
more convenient to know an interval estimate. 

The distribution of the criti cal point in re sponse surface methodology was studied by 



Diaz Garcia and Ramos-Quiro ga ( 200 ll 120021 ). when 2/(x) is defined as an hyperplane 



^In the context of the mathematical statistical the sensitivity analysis consist in to study the ways in 
which the estimators of certain model are affected by omission of a part icular set of varia bles or by the 
inclusion or omission of a particular observation or set of observations, see lChatteriee and Hadi (1988i ). 



In this work we give a response to these two questions. First, in Section [2] some notation 
is established. In Section [3] the response surface optimisation problem is proposed. First- 
order and second-order Kuhn- Tucker conditions are stated characterising the critical point 
in Section 21 Finally, the asymptotic normality of a critical point is established in Section 



2 Notation 



A det ailed discussion of respo nse surface methodology may be found in iKhuri and Cornell 
(|l987l ) and lMvers et ali (|2009l ). For convenience, their principal properties and usual nota- 
tion shall be restated here. 

Let N be the number of experimental runs. The response variable is measured for each 
setting of a group of n coded variables (also are termed factors) xi, X2, . . . , x„. We assume 
that the response variable can be modeled by second order polynomials regression model in 
terms of xf . Hence, the response model can be written as 



X/3 



(4) 



where y G K^ is the vector of observations on the response variable, X G 5ft^^'' is an 
matrix of rank p termed design or regression matrix, p = 1 -I- n -|- n{n + l)/2, (3 ^ W is 
a vector of unknown constant parameters, and e £ 5?^ is a random error vector such that 
£ '^ A/Ar(0, cr^Ijv), i.e. £ has an N -dimensional normal distribution with E(£) = and 
Cov(£) — (T^Iat. Then we have: 

. X — (xi,2;2, . . . ,x„) : Vector of controllable variables or factors. Formally, to each factor 
A,B,... is associated an x^'^ variable 

. f3: The least squares estimator of 

/3 — (/3o,/3l, ■ • ■ ,/3n,/?ll7 • ■ • ,/5nr!,/?127 • ■ • ,/5(n-l)n) 

given by 

f3 = (X'X)"^X'y = (/3o, A, . . . ,/3„,/3ii,. . . ,^„„,/3i2,. . . , /3(„_i)„)'. 
Furthermore, under the assumption that e ^ 7Va'(0, ct^Iat), then /3 ~ Mp [f3, cr^(X'X)^^] 

. 3i = (^i,...,^„)'and 



B= - 

2 



f 2^11 ^12 
^21 2^22 



/3l„ 
^2„ 



\ 



\ f3nl /3n2 

y(x) = z'(x)3 



^Pnn J 



n 



n n 



n 
i—1 i—1 j>i 



= /3o + /3ix + X Bx : 
Estimated response surface or predictor equation at the point x. 



v' (1 — XfX'X'l^-'^X'W 
. a^ = : Estimator of variance a^ such that (N — p)a^ /a^ has a y^- 

N-p \ fl I A 

distribution with [N —p) freedom degrees; denoted this fact as {N ~p)a'^ /a'^ ~ Xn_ ■ 

Finally, note that 

C^(3)^ct2(X'X)-i. (5) 

3 Response surface optimisation 

For convenience, those concepts and notations required here are listed below in terms of the 
es timated model of respons e sur face. Definiti o ns an d details properties, etc., may be found 
in lKhuri and Cornelll (|l987l) and lMvers et al\ (|2009f) . 



Ideally, the goal would be to solve (j3|), however note that in general the parameters of 
form /Jj" are unknown. Then proceeding as it is indicated in the previous section, the /S." 
parameters are estimated. Then, the response surface optimisation problem in general is 
proposed as the following substitute mathematical program 

min y(x) 

X 

subject to (6) 

X € X, 



which is a nonlinear mathematical problem, see lKhuri and Cornelll ( 1987^ . and iRaol ( 1979I ). 



Here X denotes the experimental region, which is defined, in general, as a hypercube 

X = {x\k < Xi <Ui, i = 1, 2, . . . , n}, 

where 1 = {li,l2, ■ ■ ■ ,ln) defines the vector of lower bounds of factors, and u = (ui ,U2, ■ ■ ■ ,Un) 
defines the vector of upper bounds of factors. Alternatively, the experimental region is de- 
fined as an hypersphere 

X={x|x'x<c2,c>0}, (7) 

where note that x'x = ||x|p and in general c is specified by the experimental design model 
used. In this paper it is considered X defined by ([7|). 

4 Characterisation of the critical point 

Let x*(/3) e K" be the unique optimal solution of program © with the corresponding 
Lagrange multiplier A* (/3) € SR. The Lagrangian is defined by 

L(x,A;3)=y(x) + A(||x||2-c2). (8) 

Similarly, x* (/3) £ K" denotes the unique optimal solution of program ([3]) with the corre- 
sponding Lagrange multiplier A* (/3) S 3ff . 

Now we establish the local Kuhn- Tucker conditions that guarantee that the Kuhn- Tucker 

point r*(/3) = x*(/3), A*(/3) e SR"^^ is a unique global minimum of convex program ([6]). 

df 
First recall that for / : Jft" — >■ SR, then — — = Vx denotes the gradient of function /. 

ox 

Theorem 4.1. The necessary and sufficient conditions that a point x* (/3) G 5ft" for arbitrary 
fixed f3 e ^^, be a unique global minimum of the convex program (d^ is that, x*(/3) and the 



corresponding Lagrange multiplier A* (/3) G 5R, fulfill the Kuhn- Tucker first order conditions 

M(x)3 + 2A(3)x 



VxL(x,A;/3) = <( or J> = 

3i + 2(B + A(3)I„)x 

VaL(x,A;3) = ||x||2-c2 < 



A(/3)(||x|p-c2) 







A(/3) > 



(9) 

(10) 

(11) 
(12) 



where 



M(x) = Vxz'(x) = 



9z'(x) 

ax 



with 



C,= 



= (0:I„:2diag(x):Ci: 
0' , 



;c„_i) e WP, 



,z = l,...,n-l, 0,er'-^J = l,...,^-l, 
x'A, 

observing that when i — 1 (i.e. j — 0) means that this row not appear in Ci; and 

( Oi \ 



A,; = 



0'. 



, G'k&W'-'^k^l,. 



. , I. 



In addition, assume that strict complementarity slackness holds at x*(/3) with respect 
to A*(^), that is 

A*(/3) >0 4^||x||2-c2=0. (13) 

Analogously, the Khun- Tucker condition © to p^ for /3 = /3 are stated next. 

Corollary 4.1. The necessary and sufficient conditions that a point x*(/3) G Jft" /or ar&z- 
irary /ised (3 G 3fiP, fee a unigue global minimum of the convex program (0) is that, x*{(3) 
and the corresponding Lagrange multiplier A* (/3) G 5R, fulfill the Kuhn- Tucker first order 
conditions 



xi(x,A;3) = 


r M(x)/3 + 2A(/3)x ] 

< or ( = '^ 

[ /3i+2(Bx + A(/3)I„)x J 


(14) 




VAi(x,A;/3) = ||x|p-c2 < 


(15) 




A(/3)(||x||2-c2) = 


(16) 




A(/3) > 


(17) 



an- 



d\{(3) = when ||x||2-c2 <Q at [x*(/3), A*(;3)]'. 



Observe that, due to the strict convexity of the constraint and objective function, the 
second-order sufficient condition is evidently fulfilled for the convex program ([S]). 

Next is established the existence of a once continuously differentiable solution to program 
([S]), see lFiacco and Ghaenii (,1982i) . 



Theorem 4.2. Assume that il3\) hold true and the second-order sufficient condition is for 
the convex program (0j. Then 

1. x*(/3) is a unique global minimum of program J^ and A*(/3) is unique too. 

2. For fi G 14(/3) (is an e— neighborhood or open ball), there exist a unique once contin- 
uously differentiable vector function 



r*(/3) 



x*(/3) 
A* (3) 



e^ 



.n+l 



satisfying the second order sufficient conditions of problem (^such that r*{f3) = [x*(/3), A*(/3)] 
and hence, x*(/3) is a unique global minimum of problem (0) with associated unique 
Lagrange multiplier A* (/3) . 

3. For 13 € Ve{l3), the status of the constraint is unchanged and A*(/3) > <^ ||x||^ — c^ = 
is hold. 



5 Asymptotic normality of the critical point 

This section considers the statistical and mathematical programming aspects of the sensi- 
tivity analysis of the optimum of a estimated response surface model. 

Theorem 5.1. Assume: 

1. For any (3 G V^{l3), the second-order sufficient condition is fulfilled for the convex 
program (0j such that the second order derivatives 

a2L(x,A;3) d^L{x,X;^) d^L{x,X;^) d^L{x,X;^) d^L{x,X;^) 



dxdx' 



dyid(3 



dxdX 



dXdK' 



dXdl3 



exist and are continuous in 



x*(3),A*(3)J GK([x*(/3),A*(/3)]') an, 

0^£(x,A;3) 
dxdx' 



is positive definite. 



2. /3j^ the estimator of the true parameter vector (3^, that is based on a sample of size N^ 
is such that 



3. \1S^) is fulfilled for (3 = (3. Then asymptotically 



^N, 



x*(/3)-x*(/3) 4aA„(0„,H) 



where the n x n variance- covariance matrix 



dx*{f3)\^(dx*if3) 



8(3 



d(3 



such that all elements of I 5x*(/3)/9/3) are continuous on any f3 G Ve{f3); furthe 



8/3 



x*(/3)'G-ix*(/3) 



M x*(/3) 



where 



G 



92l(x,A;/3) 



2 B-A*(/3)I, 



9xi9x' 

Proof. According to Theorem 14.11 and Corollarv l4.H the Kuhn- Tucker conditions (|9|)- (fT2|l at 

x*(3),A*(3) and to at ([Ill)-([ni) [x*(/3), A*(/3)]' are fulfilled for mathematical programs 

([3]) and (O, respectively. From conditions (fT4|) - (IT7l) of Corollary 14. 1[ the following system 
equation 



M(x)/3 + 2A(/3)x 
Vxi:(x,A;/3)= <( or 

/3i+2(Bx + A(/3)I„)x 

VAi(x,A;/3) = ||x 



2-c2 



= 

= 



(18) 
(19) 



191) with respect to x and A at 



has a solution x*(/3), A*(/3) > 0,(3. 

The nonsingular Jacobian matrix of the continuously differentiable functions (|18p and 

x*(3),A*(3) 



2(B + Ax) 2x 
2x' 



/ d^L{^,\-3) 


92L(x,A;3) \ 


dxdx' _ 


dXdx 


92L(x,A;/3) 


» / 


V dx'dX 



According to the implicit functions theorem, there is a neighborhood V^{(3) such that for 
arbitrary /3 e Ve{l3), the system ((T5)) and ((T^ has a unique solution x*(/3), A*(/3), /3 and 
by Theorem 14.21 the components of x*(/3), A*(/3) are continuously differentiable function of 
/3, see lBigelow and Shapiro! (|l974l) . Their derivatives are given by 



/ gx*(3) \ 
a3 

\ dp j 



I a2i(x,A;3) 


a2L(x,A;3) \ 


/ a2L(x,A;3) 


9x9x' _ 


aAax 


^/ 


92L(x,A;/3) 




dyidfi 


\ 9x'aA 


' J 


\ 



2(B + Ax) 2x 
2x' 



M(x) 




(20) 



The explicit form of (9x*(/3)/9/3) follow from ([20l) and by formula 
-1 



P Q 
Q' 



[I P iQ(Q'P iQ) iQ']P 1 P iQ(Q'P-iQ)-i 
(Q'P iQ) IQ'P-I -(Q'P iQ) 1 



where P and S are symmetric a nd P is non singular. 

Then from assumpti on 2 an d Rao fl973, (iii), p. 388) and lBishop et al\ ( 199ll . Theorem 
14.6-2, p. 493) (see also lCramer (1946 , p. 353)) 



'N, 



x-(«-x-(« 4m. o,J^i:<«ls ''*'•(« V 



V 9/3 



9/3 



(21) 



Finally note that all elements of (9x*/9/3) are continuous on V"e(/3), so that the asymptotical 
distribution ([2T|) can be substituted by 



'N„ 



x*(/3)-x*(/3) 



ANAojm^w-*^^^ 



df3 



dl3 



see 



Raol (|l97l (iv), pp.388-389). 



D 



D 



Now, consider that A(/3) = that is, ||x|p — c^ < at x*(/3). Thus in this case we have 
an explicit expression for x*(/3), furthermore 

Hence: 

Corollary 5.1. Assume: 

1. For any (3 G Ve{(3), the second-order sufficient condition is fulfilled for the convex 
program (0j such that the second order derivatives 

d'^yjyj) 9^y(x) 
9x9x'' 9x93" 

exist and are continuous in x*(/3) £ V^(x*(/3)) and 

a2y(x) 



9x9x' ' 



is positive definite. 



2. f3jj the estimator of the true parameter vector /3^ that is based on a sample of size N^ 
is such that 

VA^(3,-/3J^AAp(Op,S), -ls = a2(X'X)-i. 



3. Then asymptotically 



'N„ 



x*(/3)-x*(/3) 



N,, 



■Nn{On,'B.l) 



where the n x n variance- covariance matrix 



=1 



5x*(/3)^/'ax*(/3) 



d/3 



d/3 



such that all elements of I 9x*(/3)/9/3) £ 5R"^^ are continuous on any j3 e V^{l3); 
furthermore 



ax*(/3) 



dxdy 



= iB-iM(x*(3)). 



Proof. This is a verbatim copy of the proof of Theorem 14.21 noting that the conditions 
p4l) -([T7 l) are simply reduced to 

M(x)/3 
Vxj/(x) = { or 



which has the solution 



(3-^ + 2Bx 



x*(/3) = -iB^ibi. D 



n 



Conclusions 

As consequence of Theorem 14.21 and Corollary 15.11 no w is feasible t o esta blish confidence 



intervals and hypothesis tests on the critical point, see lBishop et al\ ()199lL Section 14.6.4, 
pp. 498-500); then it is possible to establish operating conditions in intervals instead of 
isolated points. 

Unfortunately in many applications, especially in industry, the number of observations 
is relatively small and perhaps the results obtained in this work should be applied with 
caution. However, the results of this paper can be taken as a good first approximation to 
the exact problem. 
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